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Summary 

A  stochastic  model  for  paired  comparisons  of  multi -attribute  social  stimuli 
is  proposed  where  one  objective  is  to  find  the  relative  importance  of  the  attri¬ 
butes  for  a  judge.  The  model  can  be  conceived  as  a  special  strict  binary  util¬ 
ity  model,  i.e.  a  BTL-model ,  and  is  related  to  the  factorial  model  of  Abelson 
and  Bradley.  The  scale  values  of  the  stimuli  are  linear  combinations  of  func¬ 
tions  of  the  stimuli's  attributes.  The  model  does  neither  assume  that  the  func¬ 
tions  are  fixed  in  advance  nor  that  different  judges  have  the  same  set  of  func¬ 
tions.  The  choice  among  such  functions,  however,  is  admitted  only  within  a 
finite  scope.  Within  the  framework  of  exponential  families,  maximum  likelihood 
estimators  and  tests  are  derived  and  applied  to  data  coming  from  two  psychologi¬ 
cal  experiments,  y 
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1 w Introduction 

Almost  all  models  of  multi-attribute  decision  anu  judgment  research  con¬ 
sider  the  (subjective)  utility  (judgment,  resp.)  U(G^)  of  an  object  (alterna¬ 
tive,  resp.)  G^  as  a  function  of  values  where  27^  is  the  aspect  of 

the  attribute  for  G^,  k=l,...,K,  and  G^  is  characterized  by  the 
with  fixed  i  £ {1 , . ,N} : 

U(G(i))  =  FLa1f1(w1{i)),...,aKf1((^^))].  (1) 

Here,  fj,...,f„  are  the  functions  which  represent,  cognitively,  the  K  attributes 
of  objects  G^,...,G^,  and  a-|,...,aK  are  the  weights  which  indicate  the  im¬ 
portance  of  the  attributes  ft  ,ft  The  attributes  jointly  describe  all 
objects  by  different  aspects.  F  is  a  real  valued  monotone  function  in  each  of 
its  arguments  and  specifies  the  composite  effect  of  the  attributes. 

If  the  f^fr,  have  scale  values  already,  e.g.  the  test  scores  of  intel¬ 

ligence,  experience,  or  neuroticism  of  an  applicant,  and  if  the  judgment  U  gets 
a  value  on  a  (subjective)  scale,  e.g.  the  degree  of  the  applicant’s  qualifica¬ 
tion,  weights  a^'s  can  be  simply  estimated  from  models,  e.g.  from  the  model  of 
multiple  linear  regression.  Clearly,  in  psychology  these  models  frequently  are 
formulated  stochastically.  In  these  cases  an  error  random  variable  should  be 
added  to  the  right  side  of  the  equation  (1). 

In  many  situations,  the  decision  between  two  objects  G^  and  at  a 
time,  which  one  of  them  is  more  attractive,  more  qualified,  or  rather  more 
threatened,  is  psychologically  more  meaningful  and  less  difficult  for  the  judge 
than  to  assign  a  number  on  a  rating  scale.  Bradley  (1976)  has  given  an  excel¬ 
lent  overview  over  the  stochastic  methods  in  the  area  of  paired  comparisons. 

The  models  being  discussed  in  this  framework  can  be  considered  as  variants  of 

p^:  =  pr(G^  is  preferred  to  G^)  =  *[U(G^),U(G^)], 

i/h  i 
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where  U  is  the  (subjective)  utility  and  ij>  is  a  combination  y  ®d  of  a  sign  de¬ 
pendent  distance  d  and  a  function  y  from  IR  to  [0,1]  which  has  the  properties 
of  a  cumulative  distribution  function.  In  Bradley's  (1976)  paper,  U(G^)  is 
most  frequently  represented  by  a  single  parameter  n ^ ,  while  representation  by 
a  function  F  is  given  for  factorial  designs.  In  factorial  designs,  the  (sub¬ 
jective)  utility  of  an  object  is  decomposed  into  contributions  from  the  respec¬ 
tive  attributes. 

In  pairwise  comparisons,  the  processing  of  the  objects  is  assumed  to  follow 
mainly  one  of  two  broad  strategies.  According  to  the  first  strategy,  each  ob¬ 
ject  G^  is  evaluated  separately  and  independently  from  ill  other  objects. 

It  then  gets  a  (subjective)  overall  value  U(G^)  of  attractiveness,  for  in¬ 
stance.  G^  ^  then  is  judged  to  be  more  attractive  than  G^ ,  whenever 
U(G^)  >  U(G^)-  According  to  the  second  strategy,  the  objects  are  compared 
attribute-wise.  In  comparing  G^  and  G^',  at  first  each  attribute  m.  k  gets 
a  value  that  indicates  whether,  and  how  much,  G^  is  more  or  less  attractive 
than  G^  with  respect  to  this  individual  attribute.  G^  then  is  preferred 
to  G^  if  the  overall  value  of  all  attribute-wise  comparisons  is  assumed  to 
favour  object  G^ . 

If  it  is  further  assumed  that,  in  the  first  strategy,  the  overall  value 
U(G^)  is  formed  by  summing  up  its  corresponding  attribute  values  or,  in 
the  second  strategy,  by  summing  up  the  attribute-wise  comparisons,  then  the 
processing  of  the  objects  for  a  decision  can  be  represented  by  an  additive  dif¬ 
ference  model  (Tversky,  1969): 

G(i)  >  G(j)  iff  J  d>kCfk(^  3  )-fk(^7  )3  >  0, 

where  ^ 


(-<0  =  -<i>kU),  and  as  well  as  ffc  are  real  valued  functions,  k=l,...,K. 


x 


Clearly,  the  additive  difference  model  represents  the  second  strategy. 

The  first  strategy,  if  represented  by  an  additive  model,  is  a  special  case  of 
the  additive  difference  model  where  all  the  difference  functions  have  the  form 


4^(6)  =  a^s.  In  this  special  case,  the  two  models  coincide,  but  not  necessarily 
the  two  processing  strategies. 

If  the  f^(j7?^)  have  scale  values  while  the  judgments  result  from  paired 
comparisons,  Srinivasan  and  Shocker  (1973)  have  proposed  an  estimation  proce¬ 
dure  for  the  weights  a^,...,aK  from  a  system  of  linear  inequalities.  This  pro¬ 
cedure  is  related  to  the  approach  of  Krantz  et  al.  (1971,  chap.  9)  for  whom 
the  main  problem  is  to  find  necessary  and  sufficient  conditions  for  the  exist¬ 
ence  of  a  specific  form  F  that  indicates  the  specific  integration  of  the  attri¬ 
butes,  if  only  a  finite  set  of  data  coming  from  paired  comparisons  is  available. 
The  first  answer  is  clear:  It  is  necessary  and  sufficient  that  scale  functions 
a,f1 ,. . .  »aKfK  exist  such  that  U(G^)  >  U(G^)  implies  that  is  preferred 
to  for  all  i  <  j.  Due  to  the  stochastic  nature  of  many  evaluation  pro¬ 
cesses,  fully  consistent  behavior  of  the  judges  cannot  be  expected.  Neverthe¬ 
less  it  seems  to  make  sense  to  estimate  the  scale  functions  or  parts  of  them, 
i.e.  the  a^,...,a^  so  that  only  as  few  as  possible  of  the  implications  above 
are  wrong,  especially  if  a  particular  form  of  For  some  special  forms  of  F 
and  f^,  k=l,...,K,  are  assumed  to  be  valid. 

However,  if  judgments  are  made  with  respect  to  paired  comparisons  of  social 
objects,  as  to  the  attractiveness  between  two  particular  persons  or  holiday 
places,  respectively,  models  for  the  estimation  of  importances  a,,...,a(<  should 
satisfy  the  following  two  conditions. 

[1 ]  It  cannot  be  assumed  that  the  attributes  of  the  objects  have  already  fixed 
functions  f^,...,fK  being  available  for  the  judge. 


4 


Under  this  condition,  the  approach  of  Srinivasan  and  Shocker  (1973)  is 
not  appropriate.  Basically,  th^  attributes  could  have  got  such  functions  from 
the  judges,  as  it  is  assumed  in  several  approaches  (e.g.  Gensch  &  Recker,  1979). 
However,  this  procedure  requires  two  conflicting  judgment  situations  within 
the  same  judge: 

(a)  the  decision  between  two  objects  (stimuli)  and  G^  at  a  time,  for 
instance,  two  holiday  places,  and 

(b)  the  decision  for  scale  values  for  each  aspect  o*  each  respective  attribute, 
e.g.  the  attracti veness  of  the  hotels. 

It  is  intuitively  clear  that  these  two  judgment  situations  will  mutually 
influence  each  other  and  thus  will  inflate  the  validity  of  the  model. 

[2]  It  cannot  be  assumed  that  different  judges  decide  on  the  basis  of  the 
same  set  of  functions  f, , . . . ,fK  for  the  attributes  ^ . . . . ^  of  the  objects. 

The  aspects  within  the  attributes  often  are  preferred  quite  differently 
by  different  judges.  For  instance,  one  sea-side  resort  will  be  preferred  by 
one  judge  due  tc  its  much  frequented  beaches  while  the  other  resort  will  be 
pref  >rred  by  another  judge  due  to  its  tranquil  beaches.  Averaging  over  judges 
would  lead  to  results  which  represent  none  of  the  judges.  Thus  an  analysis  of 
the  judgments  of  the  individual  judge  should  be  performed. 

The  objective  of  the  present  paper  is  to  propose  a  stochastic  model  for 
the  decision  process  as  well  as  for  the  estimation  of  the  weights  a^  of  the 
attributes  k=l,...,K.  Its  applicability  will  be  demonstrated  with  pair¬ 
ed  comparisons  in  two  psychological  experiments.  The  final  form  of  the  model 
satisfies  both  conditions  [1]  and  [2]. 

In  the  second  section,  normal  equations  for  the  maximum  likelihood  estima¬ 
tors,  and  a  necessary  and  sufficient  condition  for  their  existence,  will  be 
derived  as  well  as  tests  for  hypotheses  with  respect  to  the  a^'s.  In  the  third 


section,  the  results  of  two  experiments  will  be  analysed  by  the  proposed  pro¬ 
cedures.  The  results  corroborate  the  importance  of  condition  [2]. 


2.  The  Stochastic  Model 

2.1.  Presentation  of  the  Model  in  its  Preliminary  Form  and  Likelihood-Equations 
Let  G^,...,G^  be  N  stimuli  (objects)  which  can  be  described  by  K  attri¬ 
butes  7n  k  e {1 ,2,. . .  ,mk>  with  fixed  m^,  k=l,...,K.  Thus  they  may  be  represent¬ 
ed  by 

. *i!n)),n  =  l,...)N,N=  n  m..  (2) 

IK  k=l  k 

Each  of  the  N(N-l)/2  pairs  of  stimuli  are  presented  in  a  random  succession 

to  a  subject  (judge)  who  has  to  decide  in  each  case  which  one  of  the  two  stimuli 

is  preferred  to  the  other  one.  These  preferences  are  assumed  to  be  given  in¬ 
dependently.  Thus  we  have  a  full  design  in  a  double  sense:  All  objects  which 
can  be  constructed  from  the  attributes  and  all  pairs  of  objects  are  at  hand. 

Let  us  further  assume  that  a  judge  prefers  object  G^  to  G^  with  prob¬ 
ability 

pr(G^  is  preferred  to  G^):  =  p .  ^ ( a ) :  = 

K  .  K 

:  =  (l+exp{-(  £  akM^-  £  a^*)})'1,  (3) 

1  _<  i  <  j  ^  N,  where  a  =  (a-|,...,aK)  are  K  unknown  real  parameters, 

=  fk(Wk  ),  i  =  1,...,N,  k  =  1,...,K,  and  f  1 , . . . ,  f  K  are  (for  a  while)  fix¬ 
ed  known  real  valued  functions. 
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It  is  easy  to  see  that 


Pij<5)  ■ 


K 

exp{  £  ak[M^-M^]i 


K 

l+exp{  £  ak^Mkl)_Mkj^} 
exp(  l  a.M^} 

_ k=1  K  K _ 

exp{  l  a.M^}+exp{  l  a.M^} 
k=l  K  K  k=l  K  K 

n  expla.M^} 

k=I  K  K 


(3' ) 


K 

n 

k=l 


n  exp{akM^}+  n  expla^.^} 


and  that 


logit  p(j(a)  log  ■  j) 


1  <_  i  <  j  £  N. 


Thus  the  model  may  be  considered  as  a  strict  binary  utility  model  (Luce 
&  Suppes,  1965),  as  a  special  version  of  the  Bradley-Terry-model  (1952) { cf .  Zermelo, 
1929),  or  as  a  particular  regression  model  (cf.  Grizzle,  Starmer  &  Koch,  1969).  If  the 
fj,...,fK  can  vary  only  with  respect  to  the  constraint 

y  Ovnla  f  s  1  /ft**  V 


(V! 


0) 


i < i <N  > 


exp{akfk(ff|£  ;)}  =  1  (or  £  V^k'^  =  °) »  k  = 

{wSi)|1<i<N} 


we  get  the  factorial  model  of  Abelson  and  Bradley  (1954)(cf.  Bradley  &  El-Helbawy 
1976).  in  this  model,  we  can  fix  the  aj,...,ak  to  nonzero  real  numbers  (e.g. 
ak  =  1,  k  =  1  , . . . ,K)  or  put  other  constraints  on  f^,...,f^. 

Let  the  outcomes  be  random  variables  X.-  e{0,l),  where  X..  =  1(0)  has  the 

*  J  I  J 

meaning  that  is  preferrred  to  (G^  is  preferred  to  G^),  1  <_  i  <  j 
<  N.  Thus  the  likelihood  function  is  given  by 


Lii. 


•-.a* 
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pr(Xij=xij,l- <J-N)  =  .n.  pij(^  ( 1  -P1  j (a) ) 


1_X1J 


exP^  l  ak  1  xiitMkl)'Mkj)]} 
k=l  K  i<j  K  * 

~  K  •  * 

n  (l+exp{  l  a.  [M^-mP^]}) 
i<j  k=l  K  K  K 

x.je{0,l},l<i<j<_N.  (4) 

Since  the  likelihood  function  is  a  continuous  function  from  IR  to  [0,1],  it  has 
a  global  maximum,  at  least  asymptotically  at  ak  =  +  <•>  or  a^  =  -  *  for  some  of 
the  a^,  k  =  1,...,K.  Moreover,  since  the  likelihood  function  is  differentiable, 
the  partial  derivations  have  to  be  zero  for  a  maximum  likelihood  estimator  from 


It  can  be  seen  immediately  that  the  following  random  vector  Z  is  a  suffi¬ 


cient  statistic  for  a^lR*: 


Z  =  (Z,,...,ZJ  , 


where  Z.  =  £  X.  k  =  1 . K  . 

*  i<j  K  K 

Moreover,  the  distributions  of  Z  constitute  a  K-parametric  exponential 

K 

family,  since  for  every  fixed  a  e  IR 

K 

pr(Z=z)  =  exp{  j[  akzk+c{a)+d(z)},  z  =  (z1,...,z|<)  (  ;  ,  (6) 

where  p  ,  c(a)  and  d(z)  depend  on  f^,...,f^  in  an  obvious  manner. 

The  parametrization  is  identifiable  if  and  only  if  for  every  k  e  U  ,. . .  ,K}, 
mk  ^  2  and  the  values  f k( 1 ) ,. . . ,f k(mk )  are  not  all  identical.  Under  this  condi- 
tion,  an  unique  maximum  likelihood  estimator  a  :=  (a-|,...,ak)  e  IR  for  a  exists 
if  the  following  system  of  equations  has  a  solution: 

I,  Cx1j-pij(a)][M^i)-M^)]  =  0,  k  =  1.....K,  '  (7) 

which  then  is  the  unique  solution  of  (7)  and  equals  to  a. 


*  ,  U’fv 
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System  (7)  constitutes  a  system  of  normal  equations  which  result  from 
putting  all  partial  derivations  of  the  likelihood  function  (4)  equal  to  zero 
and  some  additional  standard  analysis.  That  (7)  in  fact  leads  to  unique  maxi¬ 
mum  likelihood  estimators  can  be  seen  more  quickly  by  realizing  that  (Zk) 

=  T  Pi j(a)  ],  k  =  1 , . . .  ,K,  and  thus  a  solution  a(z)  of  (7)  satisfies 

the  canonical  conditions  for  exponential  families: 

^a(z) ^k^  =  zk ’  ^  =  (8) 

(cf.  Bickel  &  Doksum,  1977,  p.  106). 

2.2.  Existence  of  Real  Maximum  Likelihood  Estimators 

-  K 

Now,  it  may  happen  that  a  maximum  likelihood  estimator  a  el R  does  c 
exist.  For  example,  consider  the  case  x^  =  1(0)  if  m|^  >  {<)  h|^  for 
i  •.  j.  In  this  instance,  a  judge  decides  (lexicographically)  with  respe 
the  first  attribute  only,  provided  that  .  The  likelihood  func..-n. 

then,  increases  with  a^  for  any  fixed  ag*.. .  ,aK  e  IR.  for  a  fixed  pair  (i,j) 
with  i  <  j,  three  cases  can  be  distinguished. 

(a)  m|^  -  Mp':  The  associated  factor  in  the  likelihood  function  is  P1-j(a), 
which  is  increasing  in  a^ , 

(b)  m|^  <  Mp^:  The  factor  is  1  -  p^a),  which  is  increasing  in  a1 ,  and 

(c)  =  Mp^  The  factor  is  constant  in  a-j. 

Thus,  if  we  want  to  speak  about  a  maximum  likelihood  estimator  in  this  case, 
the  first  component  has  to  be  «  . 

To  handle  this  problem  in  general,  let  us  state  the  following. 

Definition.  A  set  {kj,...,k^}  c  { 1 ....  ,K}  is  called  a  determining  set  (of  in¬ 
dices  of  attributes),  if  there  are  nonzero  real  numbers  a^,...,a^,  such  that 
for  all  i  <  j , 


it 
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?  a,M,(i)  >  (<)  l  a£M<j) 


-1  A 

4  =  1  4 


implies  x 


ij  = 


4=1  '  ”4 

A  determining  set  is  called  minimal  if  no  proper  determining  subset  exists, 
and  it  is  called  maximal ,  if  it  is  not  a  proper  subset  of  a  determining  set. 

Let  us  now  state  three  properties  of  those  determining  sets: 

First,  if  a  determining  set  exists  then  there  is  a  unique  minimal  and  a 
unique  maximal  one.  The  existence  is  obvious.  To  demonstrate  the  uniqueness, 
let  us  consider  two  different  minimal  determining  sets,  say,  x~  {k^,...,k.  } 
and  x  ~  {k-|  *• ..  ,k|\'r.  If  \  c  X  o r  x  cz  X  >  minimality  is  contradicted.  If 
X  \  X?  0  and  x  \  X?  0*  a  pair  (i,j)  of  objects  can  be  found  (full  design)  with 


mP  ^  for  k  €  Y  n  Y  .  and  with  [  a  [m[  ^-mP  ^  ]  >  0  and  V  « .  [Mr  ^-mP  ^  ]  <  0, 

k  k  4=1  a  ke  ke  4=1  k  kf  k4 


so  that  x..  =  1  and  x. .  =  0,  a  contradiction.  The  proof  for  maximal  sets  is  the  same. 
1  J  1  J  * 


Secondly,  there  is  a  (unique)  hierarchy  of  those  minimal  determining  sets. 


For  if  Xt  ~  {kPP...,kPp  is  the  unique  minimal  determining  set,  let  us 

L1  lt 

eliminate  all  pairs  (i.j)  with  l  PPpMpP-MpP]  f  0.  The  remaining  pairs  include, 

4=1  * 


what  can  be  considered  as  a  full  design  from  the  attributes  not  from  with 


k€Y, 


nk  observations  per  pair:  the  set  of  all  pairs  (i.j)  with 


(i)  .  JJ) 


for 


every  k^Yj  .  Clearly,  again  a  unique  minimal  determining  setY2  =  '<ki 
may  exist  with  Y2  HY|  t  0  and 

P  ^])CM{ =  0  and  {  >  (<)  0 

t  =  1 


(2) 


l2 


JD'V1)' 

k  4  k4 


4  =  1 


i.(2)  V  (2) 

k  4  k4 


imt  A.j  =  1(0). 


This  process  may  be  continued  until  the  last  minimal  determining  set  x  p  has 
been  found. 


Thirdly,  <1,2,. ...IQ  \  y  X^  Is  the  non  determining  remainder, denoted 
1  ,  r 

r*l 
p 

stochastic  remainder,  and  y  x  is  the  unique  maximal  determining  set. 

-  r=l  r 


Both  sets  may  be  empty. 
P 


Let  u  x 
r=l 


'  >  1  .  >  2  •  •  •  •  •  >  q  ^  so  that  -  (  y  » •  •  • » Y  ^  } »  ^2  -  *  Y  ^  -j » ■  •  • , 

f.  ..  ■  and  so  on.  It  is  possible  to  select  u,,...,a.  ,  a.  ,,,... ,a.  ..  ,...,a  , 
L,+L2  1  L1  L1+1  L1+L2*  q’ 

where  the  .1,  from  the  next  following  minimal  determining  set  are  smaller  than 

n 

those  from  the  preceding  one  and  where  [  a  [M^-M^]  >  (<)  0  implies  x.. 

;=1  1  yi  yi  1J 

=  1(0).  It  is  maximal  since,  otherwise,  a  further  minimal  determining  set 


would  have  been  found. 


The  Main  Result.  The  maximum  likelihood  estimators  a^,...,a^  exist  (as  real 
numbers)  if  and  only  if  there  is  no  determining  set. 

Proof.  If  there  is  a  determining  set,  then  there  is  a  minimal  one,  say, 

X  =  tk-|,...,kL}  with  The  part  of  the  likelihood  function  (4)  associ 

ated  with  pairs  (i,i)  with  i  <  j  and  (*):  \  a  [M^ 1  f  0,  can  get  close 

1=]  £  Ki  Ki 

to  1,  but  cnly  if  the  |a,  |  are  sufficiently  large,  l  =  1,...,L.  For  instance, 

let  ak  =  ttoj,  1  =  1,...,L,  and  a^,  k  4  X  ,  be  arbitrary  real  numbers.  Then, 

K  x .  1 -X • • 

n  p..(a)  1J(l-p-  -  (a))  1,  and  the  product  is  unequal  to  1  for 

i  <j ( *)  1J  ‘  1J  " 

V 

every  a  £  IR  .  The  remainder  of  the  likelihood  function  is  not  affected  by  a 
change  of  ri. 

To  prove  the  opposite  direction,  let  us  begin  with  the  assumption  of  non¬ 
existence  of  acIRK.  Let  (ajn^ , . . .  ,a^  ,aL+1 ,. . .  ,aK) ,  n  =  1,2,...,  with 
)a(n)  >  =»  ,  i  =  1 ,. . .  ,L,  be  a  sequence  which  tends  to  the  location  of  the 

maximum,  which  exists  at  least  in  IR  .  In  this  case  for  i  <  j , 


since,  otherwise,  one  factor  of  the  likelihood  function  tends  to  zero. 

There  exists  a  k  €  {1 ,. . .  ,L >  such  th  *  lim  a^/|a^  |  =  a  €  ]R  ,  say, 

n-»®  ^ 

e  =  1,...,L.  Then  the  first  equation  in  (**)  is  implied  by  f  4 

_  _ 1  x.  V.  M. 

Jl  —  1 

>  (<)  0,  i.e.,  the  subset  x  =  U |a? ^0,£=1 , . . .  ,L}  is  a  determining  set,  and 
therefore  the  proof  is  completed. 

If  there  is  a  determining  set,  then  there  is  a  maximal  one:  {yj,...,y  }, 

say.  In  this  case  the  a.,  k  ^  {y, ,. .. ,y  },  may  be  estimated  from  the  paired 

K  q  ^ 

comparisons  at  those  (i,j)  with  )  a  [M^-M^]  =  0,  i.e.,  in  a  reduced 

*=1  ri  yi 

model.  The  a^,  k  ^ {y^ , . . .  .y^} ,  which  are  found  in  this  way,  together  with 
a ,  •  ,  t  €  {y, .  ,y  } ,  may  be  interpreted  as  a  solution  of  the  problem  where 

*■  1  q 

a£,  { y i , . . , * Yq }  are  taken  from  the  maximal  determining  set.  In  this  way, 

the  model  is  also  suitable  if  a  judge  at  first  decides  lexicographically  with 
respect  to  certain  attributes,  and  only  in  cases  where  no  conclusion  can  be 
made  in  this  way,  decides  in  a  non-determini  Stic  manner. 

2.3.  Remarks. 

(a)  Obviously,  the  form  of  a  solution  of  system  (7)  depends  heavily  on  the 
form  of  the  given  f^,...,f^.  It  cannot  be  given  explicitly  in  general  but  has 
to  be  evaluated  in  every  concrete  situation  numerically. 

(b)  If  not  all  N(N-l)/2  pairs  of  stimuli  but  only  a  fixed  subset  of  them  are 
presented  to  the  subject  and  judged  by  her  or  him  similar  results  can  be  derived. 
The  only  thing  that  changes  in  the  formulas  is  that  all  sums  and  products  in 
(4),  (5),  and  (7)  have  to  be  restricted  to  this  subset.  However,  the  identifi- 
ability  of  the  parametrization  as  well  as  the  results  concerning  the  existence 
of  a€R  depend  now  additionaly  on  this  subset. 

(c)  If  the  N(N-l)/2  judgments  are  collected  independently  front  R  subjects  who 
can  be  assumed  to  judge  according  to  (3)  with  a  common  a,  or  are  collected  from 


one  subject  who  is  assumed  to  give  her  of  his  judgments  independently  R  times, 
respectively,  our  results  derived  so  far  hold  analogously.  One  only  has  to  re¬ 
place  the  X . . 1  s  and  x^.'s  throughout  by  the  arithmetic  means  of  the  correspond- 

•  J  *  J 

ing  responses  and  to  pay  attention  to  the  fact  that  then  (4)  is  no  longer  the 
likelihood  function,  but  now  its  R-th  root.  Especially,  the  model  can  be  examin¬ 
ed  with  generalized  (maximum)  likelihood  ratio  tests,  in  two  directions,  which 

2 

can  be  established  by  use  of  the  well  known  x  -approximation.  The  first  tests 
the  hypothesis  H:  "a^. .  .=aK=0"  (pure  randomness)  versus  K:  "model  (3)",  and 
significance  supports  our  model.  The  second  may  test  the  hypothesis  H‘:  "model 
(3)"  versus  K’:  "Bradley-Terry  model  (1952)",  and  non-significance  then  sup¬ 
ports  our  model.  Of  course,  one  should  clearly  distinguish  between  the  mean¬ 
ing  of  "supports"  in  the  two  tests  due  to  the  well  known  unsymmetry  in  the 
theory  of  testing  hypotheses.  It  should  be  pointed  out  also  that  in  the  case 
of  R  =  1,  both  tests  are  not  at  hand  and  even  the  parameters  in  the  Bradley- 
Terry  model  (1952)  are  then  mostly  not  estimable  within  the  maximum  likelihood 
approach. 

2.4.  The  Final  Form  of  the  Model 

Instead  of  proceeding  along  the  lines  as  indicated  in  Remark  (c),  we  pre¬ 
fer  to  generalize  our  model,  admitting  a  more  individual  behavior  of  the  single 
subject.  By  this  we  do  not  only  mean  that  parameters  a  may  now  differ  from  sub¬ 
ject  to  subject  but  also  that,  instead  of  having  f  :=  (f^,...,f^)  fixed  known, 
we  assume  now  that  only  f^  €  3k  holds,  where  3k  is  a  given  finite  set  of  real¬ 
valued  functions,  k  =  1,...,K.  Maximum  likelihood  estimates  f  :=  (f-j,...,fK) 
and  a  for  a  single  person  are  determined  now  in  the  following  way:  For  every 
fixed  f  with  fk  €  3 k  ,  k  =  1,...,K,  a  solution  of  (7)  has  to  be  found  and,  to¬ 
gether  with  f,  to  be  inserted  into  the  likelihood  function  (4).  The  largest 
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value  of  (4)  then  determines  the  maximum  likelihood  estimates  f  and  a. 

Now  let  us  assume  that  there  is  a  determining  set  x  with  respect  to  a 
special  choice  f^  €  3^,  k  =  1 To  give  meaningful  maximum  likelihood 
estimates  in  this  case  we  propose  the  following  way.  Let  us  define  the  maxi¬ 
mal  determining  set  x  max  =  Iyj.  •••»>}  in  this  case  as  to  be  the  largest  pos¬ 
sible  determining  set  under  all  possible  choices  of  f k €  sk,  k  -  Before 

we  prove  the  existence  and  uniqueness  of  x  ~  »  which  requires  a  comparison 

m3  x 

of  all  maximal  determining  sets  associated  with  f  =  (f-j , . . .  .fJ  ,  fk£3k, 

k  =  1 . K,  it  should  be  pointed  out  (as  will  be  demonstrated  later  in  Section  3) 

that  in  practice  this  work  needs  not  to  be  done  for  all  sets  of  data. 

To  prove  that  x  max  exists  uniquely,  let  us  consider  two  different  candidates 

x  max  ■  (11 . y  a"d  X  max  '  rtl . V’  sa*'  Let  us  assunie  that  Xmax^max) 

belongs  to  a  choice  gk(gk)  Uj.l  -  1 . K.  If  X  <o  or  *  MIc,  m,„. 

we  can  discard  the  smaller  one.  If  max\^ max  f  0  and  *  maxima x  f  0’  we 
can  find  a  pair  (i,j)  of  objects  (full  design)  with  ^  ^  ^  for 

k**max  ^  ^  max’  and  with 


1  an[gv  (r)^)-gv  (*iiJ/)]  >  o  and 


(j). 


p=l 


P  T 


ajg-  (r>  i^)-g-  in  f^)]  <  o. 


L.  r\  - 

>=1  P  > 


so  that  both  x..  =  1  and  x..  =  0  must  hold  for  this  pair,  which  forms  a  contra- 

’  \)  i  J 

diction.  Thereby  ap€F,  p  =  l,...,q,  and  dp  g  IR  ,  p  =  l,...,q,  are  constants 
as  described  in  the  third  property  of  the  determining  sets. 

So  let  X  max  =  be  the  maximal  determining  set,  being  construct¬ 

ed  from  all  possible  choices  of  fk  e  3  k,  where  gk,  k  =  1 ,. . .  ,K,  and  up,  p  =  1 , 
...,q  are  the  associated  choices  of  functions  and  constants  as  given  above.  In 

this  case  it  makes  sense  to  put  f  -  g  and  a  =  a  •  ,  p  =  1 .  ,q,  since 

yP  rP  yP  P 

for  this  choice  of  f  ,  p  «  all  factors  of  the  likelihood  function 

YP 


•V, 
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which  arc  related  to  comparisons  of  objects  (i.j)  associated  with  deterministic. 

decisions,  i.e.  with  /  ^  0,  tend  to  1  If  a  •»  and 

e=l  *  Y*  p 

a  /a,  =  /«  ,  p,  r  =  l,...,q.  fk  and  a^,  k£  *  ,  may  now  be  estimated 

ip  >  r  P 

from  all  comparisons  (i.j)  with  |  a  =  0.  These  estimations  are 

t=l  *  Y  t  Yt 

unaffected  by  fk  and  ak,  k  €  ^"max- 

It  is  possible  to  think  about  infinite  sets  3k  ,  which  in  general  will 
lead  to  a  better  fit  of  the  model  to  real  data.  One  may  take 
(2k>|  n 

3k  =  *fk  '0k  £  PfcClR  }>  with  a  sufficiently  differentiable  parametnzation, 
and  try  to  estimate  not  only  the  weights  a  but  also  the  parameters  e.  ,  k  =  1, 
at  a  time.  We  have  found  two  negative  effects  from  this  way.  First, 
the  distribution  of  the  total  observation  (X.j. ,l<i<j<N)  or  of  the  statistic  Z 
does  not  longer  form  an  exponential  family  in  general.  Secondly,  the  numerical 
evaluations  become  much  more  complicated  (which  is  in  part  an  effect  of  the 
first  point).  As  the  applications  will  reveal,  finite  g^'s  are  not  a  very 
severe  limitation. 

In  this  context,  the  relation  between  our  model  and  the  model  of  Abelson 
and  Bradley  (1954)  can  be  seen  more  clearly  though  both  objections  do  not  hold 
for  this  model.  If  we  put 

l  expfa.M^}  =  1  (or  £  =  0,  a.  =  1), 

k  =  1,...,K,  as  the  single  restriction  to  the  ak  and  fk>  k  =  1,...,K,  we  arrive 
at  the  model  of  Abelson  and  Bradley  (1954).  Generally,  the  g  k's  cannot  be  assum 
ed  to  be  finite,  except  for  the  instances  where  mk  <2,  k€{l,...,K}.  In  these 
instances,  the  fk's  may  be  fixed  (g  ^  =  {fkJ)  since  tfle  ak's>  k  =  are 

still  unconfined.  For  mk  >_  3  this  is  no  longer  possible  since,  by  letting  vary 
ak,  one  cannot  change,  e.g.,  the  relation  between  the  differences  of  the  a^M^, 


i  =  1,...,N,  which  are  unrestricted  in  the  model  of  Abel  son  and  Bradley  (1954). 
In  our  model,  however,  there  is  only  a  finite  number  of  possible  relations 
according  to  the  finite  3  k's.  If  m^  2,  k  =  1,...,K,  each  of  the  two  models 
is  evenly  transformable  into  the  other  one. 

Since  in  our  model,  P.jj(a)  depends  on  the  M^,  k  =  1,...,K,  n  =  1,...,N, 
only  by  their  differences  -  M^,  i  <  j,  k  =  1,...,K,  the  restriction  de¬ 
scribed  above  can  be  reached  by  adding  a  constant  bk  to  each  ,  n  =  1,...,N, 
k  =  1 , . . . ,K. 

In  their  (1976)  paper,  Bradley  and  El-Helbawy  suggested  an  algorithm  to 
maximize  the  likelihood  function  even  for  factorial  designs  as  in  the  Abelson- 
Bradley  model.  El-Helbawy  and  Bradley  (1977)  proved  the  convergence  of  this 
algorithm,  provided  the  assumption  of  Ford  (1957)  holds: 

"In  every  possible  partition  of  the  N  objects  into  two  nonempty 
subsets,  some  objects  in  the  second  set  has  been  preferred  at 
least  once  to  some  object  in  the  first  set."  (cf.  Zermelo,  1929). 

It  is  clear  that  the  nonexistence  of  a  determining  set  follows  from  the  Ford 
assumption,  but  the  converse  is  not  true:  In  the  first  psychological  experi¬ 
ment,  to  be  discussed  in  Section  3,  in  only  one  of  36  cases  a  determining  set 
exists  (and  can  be  handled  by  our  model),  but  in  28  of  the  36  cases  the  Ford 
assumption  does  not  hold. 

2.5.  A  Test  for  the  Model. 

As  long  as  the  sizes  of  the  sets  3-j  ,  ...,3k  are  not  too  large,  the  fol¬ 
lowing  single  tests,  for  every  fixed  f,  can  be  combined  to  a  simultaneous  test 
for  H:  "a^=. . .=aK=0".  Its  p-value  can  be  bounded  from  above  in  the  usual  man¬ 
ner  with  the  help  of  the  single  test's  p-values  and  Bonferroni's  inequality. 
Thus,  let  f  be  fixed  for  a  moment  and  let  us  look  for  a  test  for  H  versus  the 


general  alternative  that  H  is  not  true.  For  every  k  <_  K},  for  the  test¬ 

ing  problem  Hk:  "ak=0"  versus  Kk:  "ak/0"  there  exists  an  lIMPU-test  who  rejects 
Hk  if  Zk  falls  outside  of  an  interval.  But  the  boundaries  of  the  interval  de¬ 
pend  not  only  on  the  level  but  also  on  the  values  of  Z-j , . . .  ,Zk_1  ,Zk+j , . . .  ,Zk 
(cf.  Lehmann  (1959)  p.134).  Thus  in  view  of  the  combinatorial  difficulties 
arising  from  f,  these  tests  are  practically  not  performable  in  general. 

Alternatively,  let  us  propose  the  following  asymptotic  tests  for  H  (instead 
of  Hk)  versus  Kk,  k  =  1,...,K.  The  single  tests  hereby  are  based  on  statistics 

l  [Xii-l/2][M^i)-M<*j)]/(l/4  l  [M^-M^]2)1/2  (9) 

i<j  J  K  i<j  K  K 

and  acceptance  regions  [(f-1  (a/2K)  .o*1  (l-a/2K)],  k  =  1,...,K,  where  4>  denotes  the 

standard  normal  cumulative  distribution  function  and  a  an  upper  bound  for  the 
level  of  the  corresponding  test.  A  sufficient  condition  for  the  asymptotic  nor¬ 
mality  of  the  test  statistics  in  a  sequence  of  models  (2)  and  (3)  (triangular 
array)  is  that  for  large  fi  the  following  terms  tend  to  0: 

max  CM^r)-MjJs)]2/  [  [M^-M^]2,  k  =  1 . K.  (10) 

r^s  K  K  '  i<j  K  k 

This  since  under  H,  the  ,  1  <_  i  <  j  <_  N,  are  independently  identically  dis¬ 
tributed  Bernoull i-variables  with  parameter  1/2,  and  therefore  the  conditions 
are  equivalent  to  the  Lindeberg  condition  (cf.  Feller  (1971)  p.  264(f)). 

To  reduce  the  loss  of  power  induced  by  the  use  of  Bonferroni's  inequality, 
other  testing  procedures  are  thinkable,  e.g.  those  which  are  based  on  the  asymp¬ 
totic  joint  normality  of  the  K  statistics  given  in  (9). 

3.  Applications  to  Psychological  Experiments 

The  model  was  applied  to  data  from  two  psychological  experiments,  each 


generating  a  3  *  3  *  2  -  design. 
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The  objects  (stimuli)  of  the  first  experiment  were  schematic  faces  with 
K  =  3  attributes:  "Mouth"  ^  €  { 1  (- — - ) ,  2  ( — ),3('-')},  "Hair"  bald), 

2(short) ,3(full ) } ,  and  "Eyes"  ^3e{l(-  -),2(*-)).  see  Fig.  1. 


Please  insert  Fig.  1  here. 


Each  pair  of  the  N  =  18  faces  has  been  presented  to  the  judge  by  project¬ 
ing  the  two  corresponding  slides  onto  a  screen  in  a  random  ordering.  The  judge 
decided  spontaneously,  which  one  of  the  two  faces  looked  more  likable,  by  cross¬ 
ing  against  one  of  two  little  boxes.  In  this  manner  N(N-l)/2  =  153  decisions 
of  the  judge  have  been  recorded.  36  subjects  participated  in  groups  of  4  in 
this  experiment. 

To  get  the  numerical  analysis  manageable,  we  restrict  ourselves  to  func¬ 
tions  f^.fg.f-j  lead  to  equidistant  scales.  With  other  words,  for  every  k, 

{Mk^  =  I  ’=1 . N>  is  3  Set  e<lu'idl’stdnt  real  numbers.  Under  this 

restriction,  the  choice  of  the  admissible  functions  ’^2 ’^3  can  be  restrictecl 
to  the  following  classes  without  further  loss  of  generality  (in  a  reasonable 
sense) : 

3  1  =  Ip.q.r)  where  p(l)  =  1,  p(2)  =  2,  p( 3)  =  3 

q(l)  =  2,  q ( 2 )  =  1,  q(3)  =  3  (11) 

and  r(l)  =  1,  r(2)  =  3,  r(3)  =  2, 

3^  =  3  i  and  3  3  =  (s }  where  s  ( 1 )  =  1,  s  ( 2)  =  3. 

For  each  fixed  k€  (1,2,3),  the  justification  is  as  follows. 

In  the  model  (2),  (3‘),  the  M^-values  appear  only  in  terms  of  differences. 
Thus  their  values  can  be  shifted  and  their  smallest  value  can  be  set  equal  to 
1.  A  y-fold  of  the  differences  can  be  compensated  in  the  parameters  ak  by  a 
change  to  ak/y.  Thus  the  largest  value  of  the  M^,  i  =  1,...,N,  can  be  set 


arbitrarily  to  a  value  greater  than  I.  Here,  we  set  It  equal  to  3.  Now  we 
arrive  at  scales  tM^J  l<i<N>  =  {1,1,3},  for  k  =  1,2,  and  {M^|Ui<N}  =  {1,3>. 
Thus,  f-j  and  f2  can  be  only  permutations  of  (1,2,3).  Analogously,  for  f3  there 
are  two  possibilities.  Finally,  a  complete  reverse  of  the  values  of  a  func¬ 
tion  fk>  e.g.  (1,3,2)  to  (3,1,2),  simply  changes  the  signs  of  all  differences 
of  the  M^,  1  i  <_  N,  and  therefore  can  be  compensated  by  a  change  of  the 
sign  of  a^.  The  estimator  a  has  to  be  evaluated  from  computations  for  all  triples 
(f1,f2,f3).  As  the  above  considerations  have  shown,  computations  for  9  combina¬ 
tions  do  suffice. 

The  sole  loss  of  generality  results  from  the  equidistantness  of  the  num¬ 
bers  1 ^ ,  1  1  i  1  N,  in  other  words,  from  setting  the  middle  value  of  these 
numbers  equal  to  2.  If  this  value  would  be  replaced  by  a  parameter  v  € [1,3], say, 
this  would  lead  to  full  generality  and  would  reveal  a  direct  parallelism  to 
the  model  of  Abelson  and  Bradley  (1954).  However,  two  more  parameters,  one 
for  the  first  and  one  for  the  second  attribute,  would  be  necessary  in  this  in¬ 
stance. 

There  are  two  possibilities  to  handle  a  lexicographic  behavior  of  subjects. 
The  first  is  to  check  the  data  for  all  possible  combinations  of  fk,  k  =  1,...,K, 
whether  there  is  a  determining  set.  This  is  rather  laborious.  The  second  is 
just  to  check  whether  the  numerical  procedure  converges  In  the  right  way  for  all 
combinations  of  fk,  k  =  1,...,K.  The  results  are  given  in  Table  1. 

Please  insert  Table  1  here 

The  estimated  functions  f.|  and  f2  obviously  confirm  our  condition  [2].  A 
comparison  of  the  a's  among  subjects  having  the  same  functions  f^  and  f2  and 
signs  in  the  ak's  demonstrates  additional  individual  behavior.  Since  t‘3  -  s 
holds  for  each  subject,  f3  is  not  listed  in  the  table. 
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A  negative  sign  of  an  ak  means  that  the  subject  shows  the  opposite  prefer¬ 
ence  fk  (fk  =  hof^  with  h{l)  =  3,  h(2)  =  2,  h(3)  =  1)  to  fk  within  the  attri¬ 
bute  k  =  1,2,3. 

The  absolute  values  of  the  parameters  a-|,a2,a3  of  a  subject  indicate  which 
relative  importance  the  three  attributes  exert  on  the  decision  process.  This 
can  to  seen  more  clearly  by  considering  the  case  where  | a ^ j  >  |aj, 

0  *  Mk^  "  Mk^  =  and  Mp^  =  Mpj)  otherwise, 

t,k,p€{l,...,K}»  i  /  p  /  k,  and  1  <  i  <  j  <  N. 

It  is  easy  to  see  now  that  here  the  answer  to  the  question  whether  p.^  is  great¬ 
er  or  less  than  1/2  depends  only  on  the  sign  of  ak[Mp^-Mp^]. 

A  negative  sign  of  an  ak  and  the  absolute  values  of  the  a^,a2,a3  are  thus 
to  be  interpreted  analogously. 

The  data  of  subject  7  reflect  a  deterministic  behavior  with  respect  to  n\  3 
and  neglect  of  r\  2  and  ^3  as  long  as  the  two  faces  in  a  pair  differ  with  re¬ 
spect  to  57!  3 .  The  order  of  preference  for  values  in  5^3  is  3(w)  more  likable 
than  the  other  two  aspects,  and  2( — )  more  likable  than  1  (■*-•).  Thus,  if  we  set 
f3  =  p,{l}  is  a  determining  set.  Clearly,  it  is  a  minimal  one  and  further  con¬ 
siderations  show  that  it  is  also  the  maximal  one  for  every  choice  of  f2-  Since 
for  ^3  =  P  33  -*■  °°  ,  all  factors  of  the  likelihood  function  tend  to  1  which 
are  associated  with  comparisons  of  faces  differing  in  their  type  of  mouth,  it 
makes  sense  to  have  ^  =  p  and  83  =  «  .  f2  as  well  as  a2>a3  have  been  comput¬ 
ed  from  all  comparisons  of  faces  which  have  the  same  type  of  mouth. 

For  subject  17,  both  q  and  r  are  maximum  likelihood  estimates  for  f2>  and 
the  sign  of  a2  depends  on  the  choice  among  them.  This  result  reflects  a  re¬ 
ject!^  of  ft;  2  3  2,  the  short  hair,  and  indifference  between  the  two  aspects 
3( ful 1  hair)  and  l{bald  hair).  Similarly,  for  subject  19,  both  p  and  q  are 
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maximum  likelihood  estimates  for  f^,  leading  to  the  interpretation  that 
=  3(full  hair)  is  most  likable,  and  no  clear  preference  to  either  2(short)  or 
1 (bald)  is  made. 

Column  "Subj."  gives  the  subjective  rank  order  of  the  importance  of  the 
attributes  for  each  subject.  They  have  been  requested  in  interviews  follow¬ 
ing  the  experiment.  For  instance,  (1,3,2)  is  to  be  understood  that  the  sub¬ 
ject  rated  \  ^  as  most  and  as  least  important  for  her  or  his  judgments. 

If  a  subject  rated  all  attributes  as  equally  important  this  is  denoted  by  a 

As  a  measure  of  tne  difference  between  the  subjective  rank  order  of  a  sub¬ 
ject  and  her  or  his  estimated  rank  order  (based  on  | a-|  | ,  j |  ,  | a 3 [ ) ,  let  us  take 
the  minimum  number  of  inversions  or  transpositions  needed  to  get  from  one  rank 
order  to  the  other  one.  For  instance,  from  (1,2,3)  to  (3,1,2)  2  inversions 
and  from  (3,1,2)  to  (1,3,2)  3  inversions  are  necessary.  There  are  9  subjects 
without  any  inversion,  14  subjects  with  one,  4  subjects  with  two,  and  3  sub¬ 
jects  with  three  inversions,  giving  a  total  of  30  subjects  who  could  give  a 

subjective  rank  order.  Among  those  subjects  who  could  give  a  subjective  rank 
2 

order,  a  x  -test  with  3  df  can  be  performed  for  the  hypothesis  of  "Randomness" 
versus  the  alternative  that  less  inversions  are  more  likely.  The  p-value  for 
these  30  subjects  is  p  =  0.0268. 

The  last  column  gives  the  upper  bounds  for  the  p-values  of  the  proposed 
test  ("Randomness"  versus  "Model")  for  each  subject.  Each  value,  by 
Bonferroni's  inequality,  is  calculated  as  the  7-fold  of  the  minimum  of  the 
p-values  of  the  single  tests,  which  are  taken  from  Owen  (1962).  It  should  be 
noted  that  for  only  8  of  the  36  subjects  (16,19,24,27,28,33,34,36)  Bradley-Terry- 
estimates  i  =  1,...,N,  exist.  That  means  that  in  most  cases,  there  are 
partitions  of  the  set  of  objects  in  two  nonempty  subsets  where  no  object  in 


the  second  set  is  preferred  to  any  object  from  the  first  set  and  that  the 
second  set  contains  more  than  one  object  (one  of  the  n.  's  may  be  zero, 
i  ,N}) . 

In  the  second  experiment,  two  variants  were  realized.  The  objects  in  both 
conditions  were  descriptions  of  ten  year  old  boys  showing  learning  difficulties 
with  K  =  3  attributes: 

(a)  "Description  of  the  student  by  his  teacher"  77;^  €  {1  (he  frequently  dis¬ 
turbs  the  lessons),  2(he  uptakes  slowly),  3(he  is  afraid  of  failing)}; "Assist¬ 
ance  with  the  homework"  j^CiHby  day  nurses),  2(by  his  siblings),  3(by  his 
mother)};  "HAWIK-intel 1 igence  score  (a  German  version  of  the  WISC)" 

57;  3  €  {1  (103)  ,2(91 ) } . 

(b)  In  this  condition,  7h  2  was  replaced  by  m  4  "EEG-diagnosis  of  a  neurologist 
G  U(mild  lability  of  the  brain  functions),  2(mild  nervousness),  3(mild  vegeta¬ 
tive  disorder)}.  A  total  of  28  subjects  (teachers  and  graduate  students  in  ed¬ 
ucation)  participated  individually  in  this  experiment,  12  in  condition  (a),  and 
16  in  condition  (b).  The  pairs  of  objects  were  printed  in  a  booklet  in  a  ran¬ 
dom  ordering.  The  subjects  decided  who  of  the  two  students  in  a  pair  would 
need  more  urgently  support  by  a  special  education. 

Due  to  its  small  diagnosticity,  %  ^  was  expected  to  be  cognized  as  the 
least  important  attribute  in  condition  (b).  Since  the  importance  of  the  at¬ 
tributes  ^^,^2*^3’  as  conditions  for  learning  disabilities,  is  viewed  con¬ 
troversially  among  the  experts,  individual  preferences  were  expected.  The  re¬ 
sults  are  given  in  Table  2. 

Please  insert  Table  2  here. 


The  suggested  procedure  for  the  numerical  analysis  of  the  estimates  ffc  and 
ak  in  the  first  experiment  was  also  applied  to  the  judgments  in  the  second  ex¬ 
periment.  As  Table  2  shows,  4  subjects  decide  1  ex  i  c  og  ra  ph  i  ca  1  1  y  with 
respect  to  and  further  two  subjects  decide  strictly  lexicographically 
with  respect  to  all  three  attributes.  Thus,  more  frequently  than  in  the  first 
experiment,  determining  sets  can  be  identified  in  the  second  experiment.  The 
results  again  corroborate  the  justification  of  condition  [2],  that  is  to  say, 
there  are  more  different  than  common  strategies  among  the  subjects. 

The  expectation  concerning  the  relative  small  importance  of  r,  4  in  condi¬ 
tion  (b)  was  tested  by  assigning  ranks  to  the  estimated  weights |ak|  for  each 

subject.  The  squares  of  the  sums  of  ranks  for  each  k  were  jointly  transformed 

2 

into  the  Friedman-statistic,  which  is  x  -distributed  with  2  df  under  the  null 

2 

hypothesis:  "Equal  importance  of  the  attributes".  The  resulting  x  =  13.625 
with  a  p-value  p  <  0.005  confirms  our  expectation. 

Numerical  calculations 

The  numerical  calculations  have  been  performed  on  the  HB  66/80  at  the 
University  of  Mainz.  The  solutions  of  the  system  of  equations  (7)  are  based 
on  a  procedure  of  Werner  (1979),  which  converges  somewhat  faster  than  the 
classical,  and  also  suitable,  Newton-Raphson  procedure. 

4_._  Pi scussion  and  Summary. 

The  present  paper  proposed  a  model  for  paired  comparisons  of  multiattri¬ 
bute  social  stimuli  which  can  be  considered  as  a  stochastic  special  case  of 
the  additive  difference  model  (Tversky,  1969),  and  has  relations  to  some  other 
models,  for  instance  to  the  BTL -model  (Luce,  1959).  It  satisfies  the  cor 
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given  in  the  introduction,  that  scale  values  for  the  attributes  are  not  avail¬ 
able,  and  that  each  judge  should  be  considered  individually,  i.e.  aggregating 
across  judges  would  misrepresent  the  individual  judgment  strategies. 

As  the  results  of  the  two  experiments  show,  judges  should  in  fact  be  re¬ 
garded  individually.  Our  condition  [2]  is  especially  important  if  the  aspects 
of  the  attributes  have  no  natural  order.  (We  prefer  to  speak  of  attributes  in¬ 
stead  of  features  since  all  the  schematic  faces  having  been  evaluated  in  the 
first  experiment,  for  instance,  have  hair.  On  the  other  hand,  if  the  schematic 
faces  could  be  separated  into  two  classes  whose  members  either  have  or  have  no 
hair,  the  term  feature  would  be  more  appropriate.) 

Intuitively,  representing  the  processing  of  schematic  faces  by  an  additive 
model  seems  to  be  not  conclusive.  Recent  approaches  on  processing  information 
in  social  perception  assume  a  partition  of  the  set  of  objects  into  classes 
which  are  best  represented  by  prototypes  (cf.  Mervis  8  Rosch,  1981).  If  the 
jucgments  ll(G^)  o>  he  schematic  faces  of  the  first  experiment  would  have  been 
guided  by  such  a  processing  strategy,  classes  of  more  or  less  likable  schematic 
faces  should  have  led  to  clear  preferences  between  the  members  of  different 
classes  and  to  indifferences  between  the  members  within  classes.  However,  the 
empirical  evidence  does  not  support  such  interpretation.  It  should  be  quoted 
that  Mervis  and  Rosch  (1981)  found  in  the  present  literature 

considerable  disagreement  as  to  whether  faces  should  be  considered 
special  holistically  perceived  objects"  (p.  105). 

First,  the  judgment  situation  of  requesting  paired  comparisons  should  be 
taken  into  account  since  it  favours  a  comparison  of  the  objects  by  their  common 
attributes  (e.g.  Tversky  &  Krantz,  1969).  Even  if  a  task  consists  of  assign¬ 
ing  schematic  faces  to  one  of  two  selected  classes  of  faces  which  are  repre¬ 
sented  by  some  typical  members,  the  similarity  between  the  schematic  faces 
and  the  classes  was  well  represented  by  a  weighting  feature  model  (Reed,  1972). 
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That  moans,  a  process  of  abstraction  requires  the  attention  mainly  to  the  in¬ 
dividual  attributes  that  best  discriminate  between  the  two  classes. 

Secondly,  when  having  subjects  judge  pairs  of  schematic  faces  coming  from 
sets  in  which  the  aspects  for  some  attributes  differ  only  quantitatively,  for 
instance  when  the  eyes  differ  only  in  their  distance,  then  the  conditions  of 
transitivity  and  independence  are  less  likely  satisfied  than  in  our  first  experi¬ 
ment  with  its  qualitative  material.  Additionally,  subjective  reports  on  the 
judgment  strategies  revealed  more  attention  to  the  individual  attributes  than 
under  the  conditions  where  the  schematic  faces  had  only  one  or  no  attribute  with 
qualitative  aspects  (Mattenklott,  1979).  Qualitative  aspects  seem  to  increase 
the  independence  of  the  individual  attributes  and  thus  to  increase  their  im¬ 
portance  when  processing  schematic  faces  into  impressions  of  likability. 

Thirdly,  for  nearly  all  of  the  subjects,  the  judgments  from  the  two  ex¬ 
periments  did  not  deviate  severely  from  the  necessary  independence  conditions. 
Rather,  the  deviations  can  be  explained  by  the  stochastic  component  of  the 
evaluation  process.  This  result  neither  suggests  another  simple  polynomial  nor 
a  classification  of  the  schematic  faces  into  likable  and  unlikable  objects  as 
a  suitable  model. 

The  arguments  for  a  processing  of  objects  with  special  respect  to  their 
attributes  do  not  intend  to  challenge  a  coding  of  information  that  can  be  de¬ 
scribed  as  a  construction  of  classes  which  are  represented  by  prototypes.  How¬ 
ever,  as  our  arguments  should  demonstrate,  task  characteristics  as  well  as  the 
type  of  objects  require  a  process  of  abstraction  which  favours  the  integration 
of  schematic  faces  ty  its  attributes. 

Obviously,  the  process  of  abstraction  should  be  more  important  when  the 
attributes  describing  a  student  with  learning  difficulties  should  be  integrat¬ 
ed  to  a  judgment  of  how  severe  his  learning  disability  is.  This  process  of 
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abstraction  together  with  the  easily  separated  attributes  of  verbal  information 
should  lead  more  distinctly  to  an  attribute-based  judgment.  The  results  of  the 
second  experiment  corroborated  this  conjecture  since  6  of  28  subjects  decided 
strictly  lexicographically  with  respect  to  at  least  one  attribute.  The  results 
likewise  reflected  the  minor  importance  of  the  attribute  as  supposed  in  the 
second  variant  of  the  second  experiment.  This  result  would  have  been  less 
probable  if  the  subjects  had  processed  the  objects  configural ly. 

On  the  other  hand,  empirical  evidence  for  the  attribute-based  judgments 
do  not  allow  strict  implications  on  the  particular  integration  rule  F.  Differ¬ 
ent  integration  rules  are  compatible  with  our  model,  for  instance  a  weighted 
attribute-wise  comparison,  an  additive  combination  of  the  attributes  that  leads 
to  a  separate  evaluation  of  the  respective  objects,  or  a  lexicographic  rule  as 
performed  by  some  of  the  subjects,  especially  in  the  second  experiment.  Follow¬ 
ing  the  line  of  identifying  the  particular  integration  rule  F,  an  axiomatic 
analysis  in  the  sense  of  Krantz  et  al .  (1971)  could  have  been  performed.  How¬ 
ever,  this  approach  contains  the  problem  of  how  to  handle  the  error  (stochastic 
remainder) . 

Acknowledgement.  The  authors  are  grateful  to  Professor  P.  H.  Schonemann 
for  several  helpful  comments. 
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TABLE  1 


Estimates ,  subjektive  rank  orders,  and  p-values  of  36 


from  the  first  experiment 


Subject 

<*, 

'*!> 

*1 

S2 

*3 

Sub  j  . 

1 

(P 

,p) 

8.7230 

2.7481 

1  .2790 

(3,1,2) 

2 

(P 

,p> 

5.5402 

1  .8673 

1  .703  5 

(2,1,3) 

3 

( P 

*q> 

4.2700 

1 .5842 

1 .5627 

(1,3,2) 

4 

(P 

>q) 

2.8581 

1.1590 

1.6735 

(3,1,2) 

5 

(P 

>p) 

3 . 2006 

1.1475 

1 .0484 

(3,1,2) 

6 

(q 

,p) 

1.8752 

1 .0757 

0.6223 

,3,2,1) 

7 

(p 

,r) 

CO 

0.8339 

-0.7444 

(3,1 ,2) 

8 

(q 

,p) 

2.2223 

0.7462 

1  .5745 

(2,1,3) 

9 

(p 

,q) 

4.6310 

1.8901 

0.4377 

(3,2,1) 

10 

<p 

*p> 

2.4133 

0.7431 

2.0370 

(1,3,2) 

11 

(p 

,p) 

6.3412 

2.4086 

0.4456 

(-,-,-) 

12 

(p 

/p) 

2.4624 

2.5545 

0.7648 

(2,1,3) 

13 

(p 

,r) 

1.6597 

-1 .9288 

1  .0031 

14 

(p 

»q) 

3.8592 

0.9703 

1  .6856 

(2,1 ,3) 

15 

(p 

»q) 

2.9308 

-0.2548 

1.4654 

(3,1,2) 

16 

(q 

,r) 

0.7244 

-C.3808 

1 .0754 

(2,1,3) 

17 

(p 

,q  or 

r) 

1 .7691 

±1.6961 

0.8527 

(-,-,-) 

18 

(q 

,r) 

1.4558 

-1 .0516 

1 .3633 

(1 ,3,2) 

19 

(p 

,p  or 

q) 

1.1351 

0.9076 

0.1316 

(2,1,3) 

20 

(p 

,r) 

1 .7985 

-0.6672 

1.2310 

(-,-,-) 

21 

(p 

,p) 

1  .8366 

1 .8763 

1 .0929 

(3,2,1) 

22 

(p 

*P) 

2.6216 

2.3006 

0.8293 

(1  ,3,2) 

23 

(p 

#P) 

2.9433 

1.9670 

Q.6170 

(2,3,1) 

24 

(p 

*P) 

1.6562 

-0.4739 

0.2743 

(3,1,2) 

25 

(p 

»P) 

2.3590 

0.6951 

0.5175 

(3,1,2) 

26 

(p 

*q) 

2.1148 

1.8883 

0.3272 

(2,3,1) 

27 

(q 

/q) 

0.8842 

1.1139 

0.3190 

(1,3,2) 

28 

(p 

»P> 

1.2671 

0.2191 

0.3805 

(3,1,2) 

29 

<p 

*p) 

2.6836 

-0.1918 

1.3614 

(2,1,3) 

30 

(p 

»p) 

2.5130 

0.3020 

0.7664 

(3,1,2) 

31 

(p 

,q> 

0.6130 

1.8514 

2.5592 

(1,2,3) 

32 

(p 

»r) 

1.3106 

-1.2606 

2.4690 

(1  ,2,3) 

33 

(p 

*q) 

0.1412 

0.4580 

-0.2010 

(-,-,-) 

34 

(r 

#r) 

-0.8450 

-1 .4463 

-0.1024 

(-,-,-) 

35 

(P 

»r) 

3.0601 

0.4474 

2.6794 

(2,1,3) 

36 

<p 

*P) 

1.6102 

0.3132 

0.4356 

(3,1,2) 

subjects 


Test 


5 

.60- 

10 

1  , 

.59- 

10 

5 

.59 

10 

9 

.54- 

10 

3  , 

.13- 

10 

4  , 

.22- 

10 

5. 

.60- 

10 

7  . 

.33- 

10 

2, 

.51 

10 

2  , 

.88- 

10 

3  . 

.82- 

10 

2. 

.88- 

10 

3  , 

.96 

10 

3. 

.13- 

10 

8, 

.71' 

10 

1  . 

.09- 

10 

2, 

.55- 

10 

1  . 

.39' 

10 

8. 

.39 

10 

1  . 

.09 

10 

4  , 

.65- 

10 

7, 

.33- 

10 

4. 

.34- 

10 

-21 
-18 
-1  7 
-1  3 
-1  6 
-1  2 
-21 
-1 1 
-19 
-10 
-20 
-10 
-9 
-16 
-1  5 
-9 
-8 
-8 
-8 
-9 
-8 
-1 1 
-14 


2.08 
3.13 
7.33 
2.66 
2.88 
8.71 
3.13 
1  .69 
1  .69 
1  .36 
7.33 
4.22 
9.54 


10 

10 

10 

10 

10 

10 

10 

10 

10 

10 

10 

10 

10 


-1  3 
-16 
-1  1 
-7 
-10 
-1  5 
-16 
-15 
-15 
-2 
-1  1 
-1  2 
-13 


TABLE  2 


Estimates  from  the  second  experiment 


Subject  ( f 1 , f 2 )  S1  a2  a3 


1 

(P,P) 

-3.0963 

-2.1347 

1 .8576 

2 

(P,p) 

1  .8510 

1  .8510 

1  .1574 

3 

(P.P) 

1  .4581 

-2.3901 

0.8868 

4 

(f*,p) 

-0.4136 

2.5793 

OO 

5 

(q.p) 

2.6174 

0.9962 

1 .3898 

6 

( r,  r) 

1  .8205 

1 .0398 

3.3550 

Variant  (a) 

7 

(P.P) 

1  .5986 

2.6369 

2.3595 

8 

(PiP) 

1  .9963 

3.7057 

1 .5508 

9 

(p.q) 

0.9250 

1  .2809 

0.4629 

10 

(q.q) 

-2.31 21 

1  .6188 

0.4488 

II1 

(q,p) 

oo 

-CO 

CO 

12 

(q.p) 

3.0755 

0.9752 

1  .7471 

(fvf2) 

A 

31 

A 

a4 

A 

a3 

1  3 

( P,  r ) 

4.41 30 

-1 .5565 

2.1746 

1  4 

(p,  r) 

-0.7567 

0.2670 

0.1928 

1  5 

(q.  r) 

-1  .0317 

-2.0087 

3.0843 

16 

(q.p  or  r) 

2.1162 

+  0.4851 

1 .0253 

1  7 

(q.p) 

-2.3604 

0.9555 

1 .7578 

1  8 

(p.p) 

-1 .5368 

0.5693 

1  .1138 

19 

(P.P) 

1  .61 56 

1  .5282 

1 .7371 

Variant  (b) 

20 

(q.q) 

3.0095 

-1  .3605 

0.4570 

OO 

21 

(q.p) 

0.5470 

1 .7768 

22^ 

(r,p) 

OO 

-0 o 

oo 

23 

(p.p) 

-1 .4268 

3.0567 

CO 

24 

(p.q) 

1  7389 

0.6341 

1 .0820 

25 

(p.q) 

1  . 31  59 

0,8184 

1  .8267 

26 

(p.p) 

-3.3090 

0.5771 

1  .8202 

27 

(p.p) 

0.4711 

0.5947 

oo 

23 

(p.q  or  r) 

-2.6879 

±0.9802 

4.0594 

1,2:  Subject  11  decided  strictly  lexicographically  with  *l3>  1^2>  ' 

and  subject  22  with  ^  ^  >  “"l  3  >  **4  • 


